Skip to content

verification: deterministic host-vitest fixture bootstrap with clear diagnostics#813

Open
brandonpayton wants to merge 1 commit into
mainfrom
gascity/kd-1mr/kd-eo3w-stabilize-verification-fixture-bootstrap-when-release-ca
Open

verification: deterministic host-vitest fixture bootstrap with clear diagnostics#813
brandonpayton wants to merge 1 commit into
mainfrom
gascity/kd-1mr/kd-eo3w-stabilize-verification-fixture-bootstrap-when-release-ca

Conversation

@brandonpayton

Copy link
Copy Markdown
Member

Purpose

Make the host vitest full gate (cd host && npx vitest run) reproducible in a
fresh (e.g. Homebrew package) worktree by giving it a deterministic,
self-diagnosing fixture bootstrap.

Problem

The gate needs two artifacts vitest's globalSetup never builds:

  • host/wasm/rootfs.vfs — getpwent + node-host-mounts tests
  • local-binaries/programs/wasm64/hello64.wasm — wasm64 tests

Tests skipIf(...) themselves when these are missing, so a run can be green
while silently skipping rootfs/wasm64 coverage. Preparing them via
fetch-binaries/build-rootfs failed ambiguously:

  1. Outside scripts/dev-shell.sh: an opaque tool 'clang' not found Rust
    linker wall (the SDK cross-toolchain and even the host Rust linker are
    dev-shell-only).
  2. Inside the dev shell: a release-cache cache_key drift silently fell
    through to a source build behind an easy-to-miss stderr warning:, so a
    cache miss looked identical to a real failure.

Change

  • scripts/prepare-vitest-fixtures.sh (new) — one documented dev-shell
    command that:
    • preflights tools, reporting a MISSING TOOL distinctly from a
      release-cache miss (exit 3);
    • classifies each rootfs input as a release-cache HIT vs MISS via
      xtask build-deps resolve --fetch-only, so forced source builds are named
      and expected rather than a silent fallback;
    • idempotently builds the wasm32/wasm64 sysroots, kernel, wasm64 hello64,
      and host/wasm/rootfs.vfs;
    • writes passed/failed/skipped outcome-list artifacts plus a summary;
    • --classify-only gives a fast "is my environment ready and what will
      source-build?" pre-check.
  • docs/agent-guidance/validation.md — new "Preparing host vitest
    fixtures" section (command + expected fixture paths).
  • scripts/fetch-binaries.sh — non-fatal SDK-toolchain-awareness warning
    so the named script's own failures distinguish a missing tool from a cache
    miss (stderr-only; no control-flow or exit-code change).

Verification (in dev shell)

  • Full bootstrap: all fixtures built — passed=28 failed=0 skipped=1.
  • Release-cache classification: 14/14 rootfs inputs HIT for ABI 16.
  • Fixture-dependent host vitest (getpwent, node-host-mounts, wasm64):
    3 files / 11 tests pass, 0 skipped (previously skipped without fixtures).

Scope / limitations

  • Verified the fixture-dependent slice, not the full ~392-test
    npx vitest run (which additionally needs root npm ci; out of this bead's
    scope).
  • The real cache-MISS branch is inspection-verified only: every input was a
    published HIT here, and forcing a miss would mean clearing the resolver
    cache shared with another active porter's live builds. The MISS branch is a
    trivial symmetric path whose trigger (non-zero --fetch-only exit) is
    confirmed by the resolver code.
  • Pre-existing, out of scope: host/vitest.config.ts uses test.poolOptions,
    removed in the now-installed Vitest 4 (deprecation warning only, run still
    passes).

Bead: kd-eo3w (initiative:homebrew-all; follow-up from kd-p3hr).

🤖 Generated with Claude Code

…diagnostics

The host vitest full gate needs host/wasm/rootfs.vfs and a wasm64
hello64.wasm fixture that vitest's globalSetup never builds. In a fresh
Homebrew package worktree the bootstrap failed ambiguously: outside the dev
shell it died with an opaque "tool 'clang' not found" Rust-linker wall, and
inside it a release-cache cache_key drift silently fell through to a source
build behind an easy-to-miss stderr warning, so a cache miss looked the same
as a real failure.

Add scripts/prepare-vitest-fixtures.sh: one documented dev-shell command that
preflights tools (naming a MISSING TOOL distinctly from a release-cache miss),
classifies each rootfs input as a release-cache HIT vs MISS via
`xtask build-deps resolve --fetch-only`, idempotently builds the wasm32/wasm64
sysroots + kernel + wasm64 hello64 + host/wasm/rootfs.vfs, and writes
passed/failed/skipped outcome-list artifacts. --classify-only gives a fast
"is my environment ready and what will source-build?" pre-check.

Document the sequence and expected fixture paths in
docs/agent-guidance/validation.md, and add a non-fatal SDK-toolchain-awareness
warning to scripts/fetch-binaries.sh so its own failures distinguish a missing
tool from a cache miss.

Verified in dev shell: fixtures build (bootstrap passed=28, failed=0); the
fixture-dependent host vitest tests (getpwent, node-host-mounts, wasm64) now
run instead of skipping — 3 files / 11 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant